5.9 μs

3.7 ms

Welcome to 18.S191 – Fall 2020!

Introduction to Computational Thinking for Real-World Problems

10.6 μs

78.0 ns

https://github.com/mitmath/18S191

416 μs

121 ns

Alan Edelman, David P. Sanders, Grant Sanderson, James Schloss

12.3 μs

& Philip the Corgi

8.3 μs

Class outline

Data and computation

Module 1: Analyzing images
Module 2: Particles and ray tracing
Module 3: Epidemic spread
Module 4: Climate change

190 μs

Tools

Julia programming language: http://www.julialang.org/learning
Pluto notebook environment

10.7 μs

Module 1: Images

26.3 μs

xxxxxxxxxx
 
4 + 4

100 ns

Data takes many forms

Time series:
- Number of infections per day
- Stock price each minute
- A piece for violin broadcast over the radio

Video:
- The view from a window of a self-driving car
- A hurricane monitoring station

Images:
- Diseased versus healthy tissue in a scan
- Deep space via the Hubble telescope
- Can your social media account recognise your friends?

4.8 ms

Capture your own image!

14.6 μs

104 ns

Enable webcam

xxxxxxxxxx
 
@bind raw_camera_data camera_input(;max_size=2000)

40.7 ms

16.7 μs

UndefVarError: grant not defined

top-level scope@Local: 1

xxxxxxxxxx
 
[
    grant             grant[:,end:-1:1]
    grant[end:-1:1,:] grant[end:-1:1,end:-1:1]
]

---

grant

MethodError: no method matching getindex(::Missing, ::String)

process_raw_camera_data(::Missing)@Other: 13
top-level scope@Local: 1[inlined]

xxxxxxxxxx
 
grant = decimate(process_raw_camera_data(raw_camera_data), 2)

---

What is an image?

8.5 μs

Albrecht Dürer:

5.5 μs

6.8 μs

An image is:

A 2D representation of a 3D world
An approximation

42.9 μs

What is an image, though?

A grid of coloured squares called pixels
A colour for each pair $(i, j)$ of indices
A discretization

2.8 ms

How can we store an image in the computer?

Is it a 1D array (Vector)?
A 2D array (Matrix)?
A 3D array (tensor)?

39.5 μs

If in doubt: Ask Julia!

Let's use the Images.jl package to load an image and see what we get

11.7 μs

xxxxxxxxxx
 
begin
    import Pkg
    Pkg.activate(mktempdir())
end

80.0 ms

xxxxxxxxxx
 
begin
    Pkg.add(["Images", "ImageIO", "ImageMagick"])
    using Images
end

16.2 s

url

"https://i.imgur.com/VGPeJ6s.jpg"

xxxxxxxxxx
 
# defines a variable called `url`
# whose value is a string (written inside `"`):
​
url = "https://i.imgur.com/VGPeJ6s.jpg"  

73.0 ns

philip_file

"philip.jpg"

xxxxxxxxxx
 
philip_file = download(url, "philip.jpg")  # download to a local file

283 ms

philip

xxxxxxxxxx
 
philip = load(philip_file)

3.7 s

xxxxxxxxxx
 
philip

102 ns

Array{RGB{Normed{UInt8,8}},2}

xxxxxxxxxx
 
typeof(philip)

84.0 ns

xxxxxxxxxx
 
RGBX(0.9, 0.1, 0.1)

10.5 ms

5.6 μs

Enter cell code...xxxxxxxxxx
 
​

56.0 ns

10.7 μs

Array{RGB{Normed{UInt8,8}},2}

xxxxxxxxxx
 
typeof(philip)

81.0 ns

According to Julia / Pluto, the variable philip is an image
Julia always returns output
The output can be displayed in a "rich" way

Arthur C. Clarke:

Any sufficiently advanced technology is indistinguishable from magic.

42.5 μs

xxxxxxxxxx
 
size(philip)

253 ns

xxxxxxxxxx
 
philip

103 ns

xxxxxxxxxx
 
philip[1:1000, 1:400]

1.6 ms

How big is Philip?

He's pretty big:

8.7 μs

xxxxxxxxxx
 
size(philip)  # Pixel indices walk down a column then move over (right) a row, just like matlab

246 ns

Which number is which?

17.8 μs

xxxxxxxxxx
 
philip

67.0 ns

So, what is an image?

25.4 μs

Array{RGB{Normed{UInt8,8}},2}

xxxxxxxxxx
 
typeof(philip)

107 ns

It's an Array
The 2 means that it has 2 dimensions (a matrix)

RGBX{Normed{UInt8,8}} is the type of object stored in the array
A Julia object representing a colour
RGB = Red, Green, Blue

46.4 μs

Getting pieces of an image

55.7 μs

xxxxxxxxxx
 
begin 
    (h, w) = size(philip)
    head = philip[(h ÷ 2):h, (w ÷ 10): (9w ÷ 10)]
    # `÷` is typed as \div <TAB>  -- integer division
end

29.3 ms

xxxxxxxxxx
 
size(head)

254 ns

xxxxxxxxxx
 
size(philip)

295 ns

Manipulating matrices

An image is just a matrix, so we can manipulate matrices to manipulate the image

17.8 μs

xxxxxxxxxx
 
[head head]

20.9 ms

7.0 μs

xxxxxxxxxx
 
[
 head                   reverse(head, dims=2)
 reverse(head, dims=1)  reverse(reverse(head, dims=1), dims=2)
]

1.9 s

Manipulating an image

How can we get inside the image and change it?
There are two possibilities:
- Modify (mutate) numbers inside the array – useful to change a small piece
- Create a new copy of the array – useful to alter everything together

544 μs

Painting a piece of an image

Let's paint a corner red
We'll copy the image first so we don't destroy the original

21.0 μs

new_phil

xxxxxxxxxx
 
new_phil = copy(head)

27.4 ms

11.4 μs

red

xxxxxxxxxx
 
red = RGB(1, 0, 0)

80.1 ms

xxxxxxxxxx
 
for i in 1:100
    for j in 1:300
        new_phil[i, j] = red
    end
end

109 μs

Note that for loops do not return anything (or, rather, they return nothing)

9.3 μs

xxxxxxxxxx
 
new_phil

71.0 ns

Element-wise operations: "Broadcasting"

Julia provides powerful technology for operating element by element: broadcasting
Adding "." applies an operation element by element

12.8 μs

xxxxxxxxxx
 
begin 
    new_phil2 = copy(new_phil)
    new_phil2[100:200, 1:100] .= RGB(0, 1, 0)
    new_phil2
end

157 ms

5.7 μs

xxxxxxxxxx
 
new_phil2

109 ns

Modifying the whole image at once

We can use the same trick to modify the whole image at once
Let's redify the image
We define a function that turns a colour into just its red component

22.7 μs

redify (generic function with 1 method)

xxxxxxxxxx
 
function redify(c)
    return RGB(c.r, 0, 0)
end

25.2 μs

xxxxxxxxxx
 
begin
    color = RGB(0.9, 0.7, 0.2)
    
    [color, redify(color)]
end

53.5 ms

17.3 μs

xxxxxxxxxx
 
redify.(philip)

218 ms

xxxxxxxxxx
 
Gray.(philip)

193 ms

Transforming an image

The main goal of this week will be to transfrom images in more interesting ways
First let's decimate poor Phil

16.8 μs

poor_phil

xxxxxxxxxx
 
poor_phil = decimate(head, 5)

635 μs

6.1 μs

Experiments come alive with interaction

We start to get a feel for things when we can experiment!

11.6 μs

xxxxxxxxxx
 
begin
    Pkg.add("PlutoUI")
    using PlutoUI
end

17.6 s

7.0 μs

xxxxxxxxxx
 
@bind repeat_count Slider(1:10, show_value=true)

42.5 ms

xxxxxxxxxx
 
repeat(poor_phil, repeat_count, repeat_count)

14.7 ms

Summary

Images are readily-accessible data about the world
We want to process them to extract information
Relatively simple mathematical operations can transform images in useful ways

12.5 μs

1.9 ms

expand (generic function with 2 methods)

48.9 μs

extract_red (generic function with 1 method)

24.5 μs

decimate (generic function with 2 methods)

55.8 μs

Appendix

14.1 μs

Package environment

11.8 μs

Camera input

13.3 μs

camera_input (generic function with 1 method)

92.9 μs

process_raw_camera_data (generic function with 1 method)

59.9 μs